Synergistic Classifier Fusion for Security Applications

Unbalanced, high-dimensional, binary classifications create challenges in a variety of systems and environments, most notably within physical security and computer security domains. The imbalance within these problems consists of a significant majority of the negative (healthy, non-intruding) class and a minority (unhealthy, intruding) positive class. Any system that needs protection from malicious activity, intruders, theft, or other types of breaches in security must address this type of problem. Given numerical data that represent observations or instances which require classification, many practitioners apply state of the art machine learning algorithms to aid in solving unbalanced classification problems. The unbalanced and high-dimensional structure of the data can trouble these learning methods. High-dimensional data poses a ``curse of dimensionality'' which can be overcome through subspace modeling and intelligent fusion. A fundamental method for evaluation of the binary classification model is the receiver operating characteristic (ROC) curve and the area under the curve (AUC), and the intelligent fusion employed ties directly with the properties of this evaluation method. This work exposes the underlying statistics involved with ROC curves and leverages these properties to create synergistic classifier fusion through rankings. Decision ROC charts are a novel illustration that augment the ROC curve to provide a more complete representation of the classifier performance. Pseudo-ROC curves, created from simulated rankings utilizing principles based on the Wilcoxon-Rank sum or Mann-Whitney U statistic, provide novel insight into the behavior of classifier rankings. The critical finding involves the unique behavior of rankings for unbalanced classification problems and methods to capitalize on this behavior to improve classifier accuracy for unbalanced problems. Arguments presented include theoretical discussion, proof of principle through simulated classifier rankings examined with a factorial design, and experimental results with actual data including host-based and network-based intrusion detection datasets.